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Executive Summary 



Authors: Janice Curry and Gloria Zyskowski, Ph.D. 
• During the four-week program, 



Austin Independent School District 

Office of Program Evaluation 

Program Description 

The goal of the 1997-98 Title I summer 
school, S.O.A.R. (Summer Opportunity 
to Accelerate Reading ), was to provide 
early intervention to accelerate literacy 
learning for students entering grades 1- 
3 in the fall of 1998. This program 
originated in response to the district 
and state goal of ensuring that all 
children will read on or above grade 
level by the end of the third grade. 

S.O.A.R. was held at three schools 
(Jordan, Oak Springs, and Palm) from 
June 8-July 2, 1998. A student was 
eligible to attend if he or she was at risk 
of being retained or was below grade- 
level in literacy skills, and was 
recommended by a teacher or principal. 

The summer school program was free to 
Title I students meeting any of these 
criteria. Other AISD students who met 
the eligibility requirements for the 
program but who did not attend a Title 
I school could participate in the 
summer program by paying a fee. Each 

S.O.A.R. campus was assigned a staff 
consisting of a principal, certified 
teachers, a librarian, a parent training 
specialist, monitors, and a secretary. 

The S.O.A.R. program used a balanced 
literacy approach to reading instruction. 
Elements of a balanced reading 
program include read aloud, shared 
reading, word study, guided reading, 
independent reading, and writer’s 
workshop. Instructional materials 
specifically designed to complement 
individual reading levels were used in 
this intensive 19-day summer program. 
The student- teacher ratio was expected 
to be 15:1 on each campus. All 
teachers who participated in the 
program were required to attend two 
days of professional development to 
train them in running records, the 
assessment instrument (Developmental 
Reading Assessment), and strategies to 
improve reading skills. 



Evaluation Design 

The Title I evaluation staff collaborated 
with teachers, site principals, and the 
program director to gather data 
concerning various aspects of the 

S.O.A.R. program. Information about 
program goals, curriculum, and staff 
training was gathered from the program 
director and the curriculum specialist. 
Quantitative and qualitative data were 
collected at each campus at the end of 
the 19-day program. For assessment 
purposes the Developmental Reading 
Assessment (DRA) was used to 
determine a pre- and post-level of 
reading 

Major Findings 

A total of 388 students were enrolled at 
the three S.O.A.R. campuses. Students 
from 37 AISD elementary schools (21 
Title I and 16 non-Title I) and 4 private 
schools participated in the program. 
The ethnicity was diverse with 47% 
Hispanic, 37% African American, 16% 
Anglo/Other, and 1% American Indian. 
Forty-five teachers from 22 AISD 
campuses taught in this balanced 
literacy program. 

Attendance 

• The average daily attendance for the 

S.O.A.R. program was 354 students. 

• The average number of days each 
student was in attendance was 16.3. 
Twenty-seven percent of students 
attended all 19 days of the program. 

• The preregistration enrollment 
totaled 619 students, which 
indicated that an estimated 35% of 
the students registered for S.O.A.R. 
did not attend. 

Assessment 

The DRA uses specially selected 
assessment texts that represent a range 
of difficulty (20 texts from Level A 
through 44). The running record is 
administered as the pre- and posttest to 
determine reading level. 



85% of all students with valid pre- 
and posttest scores (n=366) 
showed improvement by 
advancing one or more reading 
levels on the DRA. 

• Only 56 students (15%) remained 
at the emergent level. Thirty of 
the 45 students who began at the 
lowest level (Level A) advanced to 
a higher level within the emergent 
level of reading. 

• Of the 85 students who posttested 
in the early stage of reading, 30 
(35%) . advanced from the 
emergent stage. 

• Of the 65 students who posttested 
in the transitional stage of reading, 
32 (49%) advanced from the early 
stage. 

• Of the 33 students who posttested 

in the extending stage of reading, 
3 (9%) advanced from early stage, 
15 (45%) advanced from 

transitional, and 15 (45%) began 
in the extending stage of reading. 

• Attending at least 17 days of the 

S.O.A.R. program had a positive 
effect on student achievement. 
When looking at all students, 
regardless of the number of days of 
attendance, the average gain was 
1.9 levels, compared with an 
average gain of 2.03 levels for 
students who attended at least 17 
days. A similar pattern was seen 
in terms of the average stage 
gains. However, because the 
average attendance rate overall 
was 16.3 days, the achievement 
differences between all students 
and those with at least 17 days of 
attendance would be expected to 
be similar. 

• When scores were examined by 
grade, students at Grade 2 showed 
the greatest mean gain in both 
level and stage. The mean gain in 
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levels was 2.4 and the mean gain in 
stages was 0.5 for grade 2. 

Survey and Interviews 

Teachers were asked to respond to a 
survey about the S.O.A.R. program. 
The principal of each campus and three 
teachers (one from each grade level) 
were interviewed. 

• Teachers expressed strong support 
for the balanced literacy approach to 
reading and indicated their 
willingness to use the strategies 
learned from the program when they 
return to their regular campuses. 

• In terms of the training provided to 
teachers, the principals agreed that 
the sessions were valuable but that 
more modeling of classroom 
behavior would have been helpful. 

• The teachers were generally pleased 
with the amount and quality of 
materials available to them for 
teaching reading. However, some 
first-grade teachers said that there 
were not enough materials to 
promote phonemic awareness skills 
for non -readers. 

• Because the DRA was very time- 
consuming, many of the teachers 
suggested assessing students with 
the DRA prior to the beginning of 
summer school to “ maximize 
instruction time . ” 

Recommendations 

As a result of reviewing the evaluation 
findings, these recommendations 
concerning the S.O.A.R. program are 
offered for consideration: 



• Continue S.O.A.R. next summer 
and incorporate elements of the 
balanced literacy approach into 
regular classroom teaching 
throughout the year. 

• Lengthen the program to 6 weeks to 
maximize the benefits of instruction 
during the summer break. 

• Maintain a low teacher-pupil ratio 
for the summer classes. 

• Enforce a stricter enrollment policy 
for students, one that stresses the 
eligibility requirements for the 
program and the importance of daily 
attendance. 

• Purchase additional materials that 
better meet the needs of pre-reading 
students. 

• Increase the emphasis districtwide 
on balanced literacy as an approach 
to teaching reading by offering 
professional development for 
teachers. 

• Recruit teachers for the summer 
program who are likely to use the 
balanced literacy training in their 
classrooms during the school year. 

• Recruit site principals for the 
program who are knowledgeable in 
balanced literacy and the mechanics 
of running a school. 

• Use S.O.A.R. teachers to train other 
teachers in balanced reading so they 
can use the strategies in their 
classrooms. 

• Increase the length of the training 
required of teachers before the 
summer program. 
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• Assess reading levels of program 
participants with the DRA before 
the summer program begins. 

• Continue to monitor achievement 
in future summer programs. 

1997-98 S.O.A.R. Budget 

The S.O.A.R. program was funded 
with Title I funds. Budget allocations 
totaled $487,620. 

Payroll Costs $250,000 

Supplies and Materials $237,620 

Program Staff 

Terry Ross, S.O.A.R. Coordinator; 

Team Leader-Language Arts 

Kathryn McKenzie, S.O.A.R. 

Curriculum Specialist 

Kathryn Stone, S.O.A.R. Facilitator 

Alicia Moore, Jordan Principal 

Ben Kramer, Oak Springs Principal 

Lisa Robertson, Palm Principal 

Debi Hyatt, Support Staff 

Gayle Short, Support Staff 
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Summer Opportunity to Accelerate Reading 

Program Description 

The goal of the 1997-98 Title I summer school, S.O.A.R. (Summer Opportunity to 
Accelerate Reading ), was to provide early intervention to accelerate literacy learning for students 
entering grades 1-3 in the fall of 1998. This program originated in response to the district and state 
goal of insuring that all children will read on grade level by the end of the third grade. 

S.O.A.R. was held at three schools (Jordan, Oak Springs, and Palm) from June 8-July 2, 
1998. A student was eligible to attend if he or she was at risk of being retained or was below 
grade-level in literacy skills, and was recommended by a teacher or principal. 

The summer school program was free to Title I students meeting any of these criteria. 
Other AISD students who met the eligibility requirements for the program but who did not attend a 
Title I school could participate in the summer program by paying a fee. Each S.O.A.R. campus 
was assigned a staff consisting of a principal, certified teachers, a librarian, a parent training 
specialist, monitors, and a secretary. 

The S.O.A.R. program used a balanced literacy approach to reading instruction. Elements 
of a balanced reading program include read aloud, shared reading, word study, guided reading, 
independent reading, and writer’s workshop. Instructional materials specifically designed to 
complement individual reading levels were used in this intensive 19-day summer program. The 
student-teacher ratio was expected to be 15:1 on each campus. All teachers who participated in the 
program were required to attend two days of professional development to train them in running 
records, the assessment instrument (Developmental Reading Assessment), and strategies to 
improve reading skills. 

The S.O.A.R. program was funded with Title I funds. Budget allocations consisted of 
$487,620, used for payroll costs ($250,00) and supplies and materials ($237,620). 

The Title I evaluation staff was asked to evaluate this summer school pilot program. A 
description of the evaluation design, a literature review, and program outcomes will be discussed in 
the following sections of this report. 

Evaluation Design 

With the assistance of Terry Ross, the S.O.A.R. program director, the Title I evaluation 
staff designed a plan for evaluating the summer reading program that would examine the 
effectiveness of the balanced literacy approach to improve reading skills in program participants. 
In addition, the effectiveness of the training program provided to S.O.A.R. teachers was 
investigated. 

The Title I evaluation staff collaborated with teachers, site principals, and the program 
director to gather data concerning various aspects of the S.O.A.R. program. Information about 
program goals, curriculum, and staff training was gathered from the program director and the 
curriculum specialist. Quantitative and qualitative data were collected at each campus at the end 
of the 19-day program. 
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Student data collected at the campus level included name, identification number, home 
school, days in attendance, and a pre- and post-level of reading as measured by the Developmental 
Reading Assessment (DRA). Information from classroom observations was used to supplement 
the achievement data that were reported at the end of the program. 

The Title I evaluation staff attended the teacher training sessions and followed up on the 
information presented to them with site visits and classroom observations at the three summer 
school campuses. During the site visits, the evaluation staff interviewed each site principal to get 
more detailed impressions of the program and how it worked at specific sites. All teachers at the 
S.O.A.R. campuses were asked to complete individual, multiple-choice surveys. The survey 
addressed the general level of staff preparedness for the summer program as a result of the specific 
training they were offered. In addition, each principal was asked to select at least three teachers to 
be interviewed by the evaluation staff for a more in-depth look at teacher perceptions of the 
program. Results of the principal and teacher interviews and the teacher surveys are included in 
separate sections of this report. Copies of the interview and survey instruments are included in 
Appendix A. 
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Review of Reading Instruction Literature 

According to most educators, there is no other skill taught in school that is more important 
than reading. It is the gateway to all other knowledge. If children do not learn to read efficiently, 
the path is blocked to every subject they encounter in their school years (National Center to 
Improve the Tools of Educators, 1996). 

The reading process is complex. According to M.R. Ruddell (1993), "Reading is the act 
of constructing meaning while interacting with text. The reader makes meaning through the 
combination of prior knowledge and previous experience, information available in text, and the 
stance he or she takes in relationship to the text. Constructed meaning is further shaped through 
social interaction and communication. ” Reading instruction builds on oral language. If that 
foundation is weak, progress in reading will be slow and uncertain (Blachman, 1991). 

The past decade has brought major breakthroughs in the knowledge of how children learn 
to read and why so many fail (Snow et al„ 1998). For most children, learning to read is a natural 
process. However, for too many children, learning to read is difficult and is associated with 
present and future difficulties. Children who do not learn to read well in the first and second 
grades are likely to struggle with reading throughout their lives (National Center to Improve the 
Tools of Educators, 1996). Teaching students to read by the end of the third grade is the single 
most important task assigned to our schools. Starting in fourth grade, schooling takes on a 
different purpose, one that is more complex and that demands higher-order thinking skills. Without 
efficient reading skills the English language, history, mathematics, current events, and the enriching 
experiences of literature and science become inaccessible to students. 

History of Whole Language us. Phonics 

The way American schools teach children to read and write has, for many years, been 
under the divergent influences of two powerful schools of thought. The two camps, whole 
language and phonics, have been competing with each other for over 30 years (Murray, 1995). 

The traditional theory of learning to read, which began in the 19th century, is that children 
learn a complex skill like reading by making sense of the smallest components of the language 
(letters) and then progressing to larger components (sounds, words, and sentences) ( Education 
Week, 1998). Children learn to read by learning to decode the language. This process begins with 
a focus on phonics (first sounding out letters, then combining letters), tightly controlled 
vocabulary, and short basal (or basic) reading passages, followed by numerous skills exercises, 
each with only one correct answer. 

The whole-language theory holds that learning to read and write English is like learning to 
speak it - a natural, unconscious process best fostered by unstructured immersion (Lemann, 1997). 
One of the central beliefs of whole language (also known as “look-say” or “sight reading”) is that 
language should be learned from whole-to-part with word-recognition skills being picked up by the 
child in the context of reading, writing, and immersion in a print-rich classroom (Levine, 1994). 
The whole-language approach stresses reading of children’s literature rather than basal readers, 
offers a supportive and tolerant atmosphere for learning to read, and emphasizes early writing to 
improve reading and thinking skills. 
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Although the current whole-language movement began in the early 1970s, the dispute 
about reading instruction began much earlier. In the mid-nineteenth century, Horace Mann 
challenged the use of phonics as the traditional method of teaching reading (Levine, 1994). He 
supported the “look-say” method that came to dominate American education during the progressive 
education movement in the 1920s. However, in the 1950s, the meaning-first approach was 
denounced in the best seller by Rudolph Flesch, Why Johnny Cant Read , and the pendulum swung 
back to phonics (Levine, 1994). 

By the early 1970s, most schools had returned to a phonics-based program. Then, as these 
programs began to be criticized by some teachers and academics as “killing off children's interest 
in reading ” through the use of worksheets, basal readers, and ability reading groups, support for 
whole-language resurfaced (Levine, 1994). Research by Marie Carbo in 1987 found that for most 
children, it is harder to learn phonics from part-to-whole (phonics first) than from whole-to-part 
(reading and writing first with phonics from words in familiar texts). The National Council of 
Teachers of English (NCTE), the International Reading Association (IRA), and numerous teacher 
groups support whole-language reading instruction (Levine, 1994). 

The whole-language philosophy gained widespread acceptance in California in 1987 when 
the state education department incorporated it into the state’s English/language- arts framework 
with the approval of whole-language based textbooks in grades K-8 (Diegmueller, 1996). 
However, California policymakers reversed their stand after the reading scores from the 1994 
National Assessment of Educational Progress (NAEP) showed that 59 percent of the state’s fourth- 
graders read below the basic level; only one state scored lower. With passage of the “ABC” law, 
California legislators now require state officials to give adequate attention to phonics in 
instructional materials (Diegmueller, 1996). 

Which of these philosophies of reading instruction will win out? Many educators believe 
that the current debate concerning the “best” approach to teaching reading will ultimately lead to a 
healthy balance between the two approaches, blending the “best of both worlds ” (Diegmueller, 
1996). Research strongly indicates that students will be the most successful if a balanced 
approach is used - teach phonics in a systematic fashion within the context of real stories 
(. Education Week , 1998). 

The Balanced Reading Approach 

The problems many children encounter in learning to read could be prevented with 
excellent instruction and an early exposure to language skills and rich literature, according to a 
report by the National Research Council (NRC), Preventing Reading Difficulties in Young 
Children (1998). The NRC, which studied a wide range of reading research over the past two 
years, called for an end to the “reading wars” that have divided educators, researchers, and 
lawmakers with opposing views of how children should learn to read. This well-received report 
recommends that children learn to read through explicit phonics instruction and by sounding out 
unfamiliar words, but it also urges daily exposure to literature and attention to comprehension. 
The teaching of reading, according to the NRC, must encompass an integration of techniques that 
develop phonemic awareness, reading fluency, and comprehension throughout early childhood, and 
cannot rely on one method of instruction. The best practice combines “immersing children in rich 
language by reading to them and providing access to a variety of texts, while explicitly and 
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systematically teaching them the sounds and their symbols and connecting these to decodable 
texts” (Adams, 1990). 

The current changes in the approach to teaching reading have left teachers with the 
dilemma of providing young readers with materials and instruction that fit their individual levels of 
development. Teachers must assess reading level so that the books children read will offer just 
enough challenge to support problem solving, but be easy enough to support fluency and meaning 
(Fountas and Pinnell, 1995). 

The components of a balanced literacy program are defined in the research-based Ohio 
State University Early Literacy Learning Initiative developed in 1984 (Fountas, 1995). A balanced 
language arts program includes a combination, or balance, of the following components, which 
together comprise a daily reading/language instructional program: 

• Reading Aloud to Children. Throughout the day the teacher reads to students a 
variety of quality literature - fiction, nonfiction and poetry. By reading to the 
students, the teacher models fluent, expressive reading and shares an enthusiasm for 
books. Reading aloud is seen as the single most influential factor in the young child’s 
success in learning to read. 

• Shared Reading. During shared reading, the teacher and students read together from a 
big book, or other enlarged text such as group-produced projects and experience 
charts. During shared reading, students learn concepts about print, vocabulary in 
context, and other reading skills. 

• Guided Reading. Guided reading provides the opportunity to work with small groups 
on books that present a challenge. From careful observations and assessment, the 
teacher determines which book would be at an appropriate level for each child. 
Groupings remain flexible so that students can move ahead as quickly as possible. 
According to Regie Routman ( Invitations , 1991), “Guided reading is the heart of the 
instructional reading program. ” 

• Independent Reading. Children read on their own or with partners from a wide range 
of materials. Voluntary self-selected reading is critical to the reading program. 

• Shared Writing. The teacher and students work together to compose messages and 
stories. The teacher supports the process as scribe. 

• Interactive Writing. Similar to shared writing, but stories are written using a “shared 
pen” technique that involves children in the writing. 

• Guided Writing or Writing Workshop. During guided writing, the teacher serves as a 
facilitator and guide to an individual or a small group of students. 

• Independent Writing. It is important to provide many opportunities throughout the 
day for students to engage in independent writing activities, such as journal entry or a 
new version of a familiar story. 

One promising program, Reading Recovery, is embraced by both phonics and whole- 
language advocates as an effective reading program (Levine, 1994). Reading Recovery combines 
quick, accessible phonics instruction with whole-language activities. The early reading 
intervention effort, designed by Marie Clay, began as a nationwide program in New Zealand in 
1979 (Pinnell, et al„ 1988). 
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Implications for Instruction 

In Preventing Failure in Early Reading Programs (1998), Bill Honig states that one of 
the most important breakthroughs in reading research in the past decades is how important being 
able to hear and manipulate the discreet sound parts of words - phonemic awareness - is to 
learning to read. Most children acquire basic phonemic awareness in kindergarten by such 
activities as rhyming and sound word games. According to Honig, about one sixth of our children 
have “ phonological wiring” problems, and without special assistance from mid-kindergarten on 
will not acquire basic phonemic awareness. Many of these students will end up in special 
education or remedial programs because they were not properly taught initially. The implication is 
that kindergarten programs must identify and assist those children who will have an extremely 
difficult time learning to read without intervention. 

Disadvantaged children coming from literature-poor homes will most likely need explicit 
instruction (Murray, 1995). Research by Barbara Foorman of economically disadvantaged second 
and third graders shows that students with low phonemic awareness suffer in whole-language 
classrooms and stay at the bottom of the class. “Whole language fuels good readers , but 
shortchanges poor readers , ” said Dr. Foorman. Estimates are that only one out of eight children 
not reading on grade-level by the end of first grade will ever read grade-appropriate materials 
without extensive and well-designed intervention programs (Honig, 1996). 

The Houston Independent School District, where 65 percent of the students qualify for free 
or reduced-price lunches, has taken an “almost militaristic stand in addressing the teaching of 
reading” (Manzo, 1998). After using the whole-language approach in Houston schools for a 
decade, a balanced approach to reading initiative was implemented. When the 1998 Texas 
Assessment of Academic Skills (TAAS) scores were released, Houston schools showed that 85 
percent of third graders, 89 percent of fourth graders, and 88 percent of fifth graders had met at 
least the minimum standards on the reading portion of the test. Fourth graders have improved 1 1 
percentage points since 1996, the year before the new mandate went into effect. 

To promote reading success for all children, teachers will need to be highly skilled in 
reading techniques and strategies (Manzo and Sack, 1997). Extensive teacher training is essential 
to children learning to read. Most elementary teachers have little formal training in the structure of 
the language and in how children learn to read. According to Robert Slavin, a researcher at Johns 
Hopkins University and creator of Success for All, “In a sense , there is an enormous retooling 
job to be done to bring teachers of beginning reading up to the current conception” of how to 
teach reading (Manzo and Sack, 1997). According to Fountas and Pinnell (1995), “Good 
teaching is the foundation of education and the right of every child. ” 

Reading Initiatives 

Political leaders have responded to poor reading scores by supporting initiatives for 
improving students’ reading skills. Nationwide in 1994, 4% of fourth graders scored at the top 
level of reading on the National Assessment of Educational Progress, and 44% scored below basic 
in reading (. Education Week , March 20, 1996). President Clinton has proposed the America Reads 
program to get students reading on grade level at the end of third grade as defined by their ability 
to meet the basic level of the NAEP ( Education Week , September 11, 1996). The Administration 
proposed legislation that would launch the first nationwide effort to supplement classroom 
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instruction in reading with high-quality volunteer tutoring, primarily after school, on the weekends, 
and during summers. According to information available on the America Reads website, Congress 
and the Administration reserved $210 million for a children's reading initiative in the 1998 budget 
to be allocated to America Reads once authorizing legislation passes and is signed into law. 

In January 1996, Governor George W. Bush announced his reading initiative for Texas. 
The goal of the Texas Reading Initiative is for all students to read on grade level by the end of 
grade 3 and to continue to read on grade level throughout their schooling. To assist school districts 
and teachers with the information essential to the success of this goal, the Texas Education Agency 
(TEA) has published a document entitled, Beginning Reading Instruction. The document 
describes the 12 essential components of a research-based reading program along with factors that 
support effective reading instruction in the classroom and on the campus. The 12 essential 
components of a successful reading program as reported by TEA state that all children should have 
opportunities to: 

• expand their appreciation for and use of oral language; 

• expand their appreciation for and use of printed language; 

• hear good stories and informational books read aloud daily; 

• understand and use the building blocks of spoken language; 

• learn about and use the building blocks of written language; 

• learn about the alphabetic principle and about the relationship between the sounds of 
spoken language and the letters and words of written language; 

• learn decoding strategies; 

• relate spelling and writing to reading; 

• practice accurate and fluent reading in readable, decodable stories; 

• develop new vocabulary through wide reading and direct vocabulary instruction; 

• read and comprehend a wide assortment of books and other texts; and 

• learn comprehension strategies and engage in higher order thinking skills as they 
reflect upon and discuss what they read. 

According to the National Center to Improve the Tools of Educators (1996), reading 
failure is not concentrated among specific groups of students. Students who have difficulty reading 
represent a ‘‘virtual cross-section of American children ” including "rich and poor, male and 
female, rural and urban, and public and private school children in all sections of the country. ” 
Thirty-two percent of fourth graders whose parents graduated from college read below basic level 
on the 1994 NAEP. According to the National Research Council (1998), “children who need 
additional support for early language and literacy development should receive it as early as 
possible. ” 

In Texas, as in the rest of the country, teaching reading presents a great challenge. This 
quote by Marilyn Haring, dean of Purdue University’s School of Education, summarizes the issue: 
“Literacy is THE most important skill we teach, and it should be taught early and well. It is the 
key to every student’s future success in school. To help our children achieve the proficiency they 
need, it is our responsibility to use any proven method available to us. ” ( Purdue News, April 
1998) 
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Quantitative Data 

As stated in the program design, this evaluation examined attendance records as well as the 
reading gains on the DRA. In addition to the analysis of reading gain data for all students, a 
separate analysis of gains by students attending S.O.A.R. 17 days or more was conducted to 
determine if number of days in attendance would affect outcomes. This section will summarize 
S.O.A.R. student and teacher demographics, and attendance and assessment data. 

Student Demographics 

Students from 37 AISD elementary schools (21 Title I and 16 non-Title I) and 4 private 
schools participated in the 1998 S.O.A.R. program. Sixty-nine students, who did not attend Title 
I schools, paid a tuition fee to attend S.O.A.R. 

Of the 388 students who attended more than five days, 55% were male and 45% female. 
The grade distribution was balanced with 25% grade 1, 37% grade 2, and 38% grade 3. The 
ethnicity was diverse with 47% Hispanic, 37% African American, 16% Anglo/Other, and 1% 
American Indian. Figure 1 presents the ethnicity for S.O.A.R. students. 

Figure 1: Ethnicity for 1998 S.O.A.R. Students 
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Teacher Demographics 

Forty-five teachers taught reading in the S.O.A.R. classrooms. The ethnicity of the 
teaching staff was not as diverse as for S.O.A.R. students; 57% were Anglo, 23% were African 
American, and 20% were Hispanic. Only two of the 45 teachers were male. 

During the regular school year these teachers teach at 22 different AISD campuses. The 
majority (93%) of the teachers taught core academic subjects during the 1997-98 school year. 
Other teachers included an elementary counselor, an elementary music teacher, and a high school 
vocational teacher. Figure 2 shows the number of teachers by grade or subject taught during the 
1997-98 school year. 
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Figure 2: Number of S.O.A.R. Teachers by Grade or Subject Taught During 1997-98 




The majority (45%) of the S.O.A.R. teachers had more than six years of AISD teaching 
experience. The average number of years of teaching experience with AISD was 8.7 years. The 
breakdown of experience teaching in AISD is as follows: 

• 0-5 years - 20 (45%); 

• 6-10 years - 6 (15%); 

• 1 1-20 years - 15 (34%); and 

• 20+ years - 3 (7%). 

The overall pupil-teacher-ratio was low, with nine students to each teacher. A ratio of 15 
students to one teacher was the original program goal. 

Attendance 

Because only students who attended five or more days were included in the count, the total 
number of students used in this evaluation is 388. Nine students were excluded from the total 
because of a teacher error in recording attendance. An additional four students were recorded as 
enrolled, but stayed less than five days. The preregistration enrollment totaled 619 students, which 
indicates that an estimated 35% of the students registered for S.O.A.R. did not attend. 

According to records from the three campuses, an average daily attendance for the 
S.O.A.R. program was 354 students. The highest average daily attendance was 386 and the lowest 
average attendance was 308. The average daily attendance at Iordan was 141, Oak Springs was 
97, and Palm was 116. The average number of days each student was in attendance was 16.3. 
Twenty-seven percent of the students attended all 19 days of the S.O.A.R. program. 
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Assessment 

The assessment instrument selected for use in the S.O.A.R. program is the Developmental 
Reading Assessment (DRA). The DRA, published by Celebration Press, is designed to provide a 
method for teachers to assess and document primary students’ development as readers over time 
within a literature-based instructional reading program. The DRA, used with kindergarten through 
third-grade students, is conducted during a one-on-one reading conference as children read 
specially selected assessment texts. The procedures used incorporate the work of Dr. Marie Clay, 
including the use of running records described by her in An Observation Survey of Early Literacy 
Achievement (Heinemann 1993). 

The DRA assessment texts represent a range of reading difficulty (20 texts from Level A 
through 44). There are four stages of literacy identified by the DRA - emergent (levels A-2), early 
(levels 3-10), transitional (levels 12-24), and extending (levels 28-44). The running record is 
administered as the pre- and posttest to determine reading level. See Appendix B for a chart of text 
book leveling correlations. 

Because the summer program was only 19 days long, it was necessary for teachers to 
assess students’ reading levels promptly so that books could be selected for each student and 
guided reading could begin. Teachers reported using from 2-5 days to do the pretest and less time 
to complete the posttest. 

Increase in Leuels 

During the four-week S.O.A.R. pilot program, 85% of all students with valid pre- and 
posttest scores (n=366) showed reading improvement by advancing one or more levels on the 
DRA. The average gain in levels for all students regardless of the number of days of attendance 
was 1.9, with a range from zero to ten levels. Figure 3 shows the number of levels advanced and 
the number of students advancing in each group for all students with pre- and posttest scores. 



Figure 3: Number of Levels Gained and the Number of Students in Each Group 
for All Students with Valid DRA Pre- and Posttest Scores 
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The greatest movement occurred at level A from pretest to posttest. Fifty-two students advanced 
from the lowest level (A) to a higher level during S.O.A.R. 

Increase in Stages 

It is more difficult to advance from one stage to another than it is to move from one level to 
another, especially if the student started at the lowest level in a given stage. In all, 111 students 
(30%) advanced one or more stages. Figure 4 shows the percent of students at each stage at 
pretest and posttest. The graph shows that the percentage of students at the two lowest stages 
(emergent and early) decreased from pretest to posttest, while the percentage of students at the two 
highest stages (transitional and extending) increased from pretest to posttest, which is evidence of 
reading gains for S.O.A.R. 
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Figure 4: Percentage of All Students at Each Pretest and Posttest Stage 
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Further analysis of the reading assessment data for all students with valid pre- and posttest 
scores reveals the following information: 

• Only 56 (15%) remained at the emergent level. Thirty of the 45 students who began at 
the lowest level (A) advanced to a higher level within the emergent level of reading. 

• Of the 85 students who posttested in the early stage of reading, 30 (35%) advanced 
from the emergent stage. 

• Of the 65 students who posttested in the transitional stage of reading, 32 (49%) 
advanced from the early stage. 

• Of the 33 students who posttested in the extending stage of reading, 3 (9%) advanced 
from the early stage, 15 (45%) advanced from transitional, and 15 (45%) began in the 
extending stage of reading. 

When scores were examined by grade, it could be seen that Grade 2 had the greatest mean 
gain in level and in stage. All students with pre- and posttest scores were included in this 
comparison. The mean gain in levels was 2.4 for grade 2, compared with 1.1 for grade 1, and 1.9 
for grade 3. The mean gain in stages was 0.5 for grade 2 and 0.2 for grades 1 and 3. Table 1 
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shows the minimum, maximum, and mean gains for DRA levels and stages by grade for all 
students with valid pre- and posttest scores. 

Table 1: Minimum, Maximum, and Mean Gains for DRA Levels and Stages by Grade 



for All Students with Valid DRA Pre- and Posttest Scores 
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2.0 
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Days in Attendance - J7+ 

To determine the effect of attendance on reading gains, the gains for students with 17 or 
more days in attendance were compared with gains for all students. By examining Table 2, it can 
be seen that attending at least 17 days of the S.O.A.R. program had a positive effect on student 
gains. When looking at all students, regardless of the number of days of attendance, the average 
gain in level was 1.9, compared with an average gain of 2.03 levels for students who attended at 
least 17 days. A similar pattern can be seen when examining average stage gains. However, 
because the average attendance rate overall was 16.3 days, the achievement differences between all 
students and those with at least 17 days of attendance would be expected to be similar. 

Table 2: Mean Level and Stage Gains for All Students in Attendance 
and for Students with 17+ Days in Attendance 



All 17+ Days 


Mean Level Gain 


I 19 


2.03 


Mean Stage Gain 


0.32 


0.35 



Increase in Leuels 

When gains are compared for the percentage of students with 17+ days in attendance and 
for all students making gains, the 0 and 1 level gains are higher for all students than for those that 
attended 17+ days. Conversely, there is a higher percentage of 17+ students making 3-4 level 
gains on the DRA. Gains of 4-10 levels are similar. Figure 5 shows the percentage of all students 
and students with 17+ days in attendance that made gains of 0-10 levels on the DRA. 
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Figure 5: Percentage of All Students and Students with 17+ Days 
in Attendance That Made Gains of 0-10 Levels on the DRA 
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Increase in Stages 

A stage consists of several levels of books. To move from one stage to another (emergent, 
early, transitional, and extending) is more difficult for students than it is to move from one level to 
another. The majority of students made no advancement to the next stage of literacy according to 
the DRA. A higher percentage of students who attended for 17+ days had one and two stage gains 
during the S.O.A.R. program. Figure 6 shows the percentage of all students and students with 17+ 
days in attendance that made gains of 0-2 stages on the DRA. 



Figure 6: Percentage of All Students and Students with 17+ Days 
in Attendance That Made Gains of 0-2 Stages on the DRA 




Stages Gained 



El All Students H 17+ Days 




18 



22 



97J0 



Summer Opportunity to Accelemte Readings 1998 



Qualitative Data 

Qualitative information is important to this evaluation because new approaches to reading 
were implemented in this summer school program. Teachers and principals who worked with the 
balanced literacy framework were thought to have valuable information to share. This section will 
include information obtained from the teacher survey and interviews, principal interviews, and 
comments from the S.O.A.R. project director and curriculum specialist. 

Teacher Survey 

Teachers at each of the S.O.A.R. sites were invited to respond to a multiple-choice survey 
about the S.O.A.R. program and the balanced literacy approach for teaching reading. A total of 45 
teachers and librarians responded to the survey. In general, teachers were very positive about the 
program. Jordan teachers were less confident about the training they received on the DRA than 
Oak Springs and Palm teachers, but were more positive about the effectiveness of the DRA as an 
assessment tool for S.O.A.R. The two survey questions with the strongest overall agreement 
showed the teachers’ support for the balanced literacy approach to reading and their willingness to 
use the strategies learned from the program when they return to their regular campuses. Table 3 
shows the mean responses to the teacher survey by campus and across the three campuses. 



Table 3: Mean Responses to S.O.A.R. Teacher Survey by Campus and Across Campuses 



Suruey Questions 


Jordan 

(n=l7) 


Oak Springs 

(n=l3) ( 




The staff development 1 received 
adequately prepared me to teach in the 
S.O.A.R. program. 


3.4 


4.7 


3.9 


3.9 


I received enough information during 
training to feel confident that 1 
administered the Developmental Reading 
Assessment correctly in my classroom. 


3.2 


4.5 


4.1 


3.9 


1 feel that the Developmental Reading 
Assessment tuas a good instrument to 
measure growth during the S.O.A.R. 


4.1 


3.9 


3.8 


3.9 


The daily schedule contained an adequate 
mix of acnuities to keep all students 
engaged academically throughout the 
day. 


3.6 


4.5 


3.8 


3.9 


The daily schedule and the curriculum 
worked together to help ensure that 
student progress was achieved in the 
S.O.A.R. program. 


4.2 


4.6 


3.9 


4.2 


1 feel that a balanced reading program 
provides useful instructional strategies to 
accelerate the literacy learning of 
students . 


4.6 


4.8 


4.9 


4.8 


1 will use strategies and/or materials 
from the S.O.A.R. program in my regular 
classroom next year. 


4.5 


4.8 


4.5 


4.6 



Note; Scale is as follows: 5=Strongly Agree : 4= Agree ; 3=Unsure; 2=Disagree; and J=StrongJy Disagree 
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Teacher Interviews 

Three teachers at each of the campuses were asked to participate in an interview about the 
S.O.A.R. program with a member of the evaluation staff. Teachers were asked to respond to 
questions about the training, daily schedule, materials, and assessment. In order to improve the 
program for next year, teachers were asked to give suggestions that might make S.O.A.R. a more 
meaningful experience for participants. Additional information was obtained from the comment 
section of the multiple-choice survey. Teachers’ comments are included in the following 
paragraphs. 

Training and Preparation 

While teachers generally agreed that the training for S.O.A.R. was beneficial, many of the 
teachers expressed a desire for additional information. More sessions on guided reading, running 
records, structuring and providing materials for centers, and writer’s workshop for the emergent 
reader were suggested as possible topics for future training. 

Some of the first-grade teachers commented that they felt that the training pertained to 
older students. One teacher stated that “grade-level breakout at the training would be helpful for 
teachers to see what the program would look like at their particular level. ” Orientation with the 
books at the training would be beneficial before going to the campus, reported several of the 
teachers. 

Two of the teachers interviewed had just completed their first year of teaching and both 
agreed that the balanced literacy training “would have made the first year go much smoother. ” 
Another teacher commented that this workshop was more effective than previous training because 
of the immediate use of information after training and the debriefing sessions. 

Daily Schedule 

The teachers from Oak Springs who were interviewed said that the schedule worked well 
in their classrooms. According to a third-grade teacher, “The schedule provides clear directions. 
Everyone knows where to go, at what time, and what to expect. ” (See Appendix C for a copy of 
the S.O.A.R. schedule.) 

The daily schedule for S.O.A.R. was academically challenging - 3 1/2 hours with no 
breaks. Many of the teachers felt that children needed some time to “move and bum off energy. ” 
One teacher wrote, “A time for children to have a short break would be beneficial to the children 
to aid their ability to focus. ” Another teacher reported that she used gross motor activities to 
release energy. Some modifications to the schedule by some teachers interviewed included 
adjusting the schedule somewhat by reducing the time for the guided reading, rearranging the order 
of components, breaking the writer’s workshop into shorter segments, or letting children choose 
free centers instead of assigning centers. 

At least once each week, teachers met with Kathryn McKenzie, the curriculum specialist, 
to address questions about the balanced literacy approach. Ms. McKenzie observed in classrooms 
and gave teachers feedback about ways to enhance the program with their students. The planning 
and debriefing time on other days was used for grade-level or other meetings. A few of the 
teachers commented that the debriefing time took away from their planning time. 
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Materials 

Each grade level at each of the S.O.A.R. campuses was provided reading materials to 
share. Each teacher also received consumable supplies to use in the classroom. For a complete list 
of S.O.A.R. materials, see Appendix D. 

The teachers were generally pleased with the amount and quality of materials available to 
them for teaching reading. One teacher at Oak Springs stated that “It is wonderful to have all the 
materials you need. I can't describe the feeling. ” However, some first-grade teachers said that 
there were not enough materials to promote phonemic awareness skills for non-readers. Several 
teachers suggested the addition of supplemental materials such as magnetic letters and dry erase 
boards for every student. One teacher commented that “students still needing letter/sound 
knowledge and who have very short memory recall need activities to help auditory and visual 
perception.” A common feeling was expressed by one teacher - “Every classroom in AISD 
should be able to have access to these kinds of materials. ” Teachers expressed regret that they 
would not have materials like these when they return to their campus. 

Assessment 

The Developmental Reading Assessment (DRA) was used as a pre- and posttest for the 
summer program. Teacher training with the new assessment was provided by a representative of 
Celebration Press, the instrument’s publisher, during a three-hour session at the S.O.A.R. staff 
development. 

The pre-assessment was reported to be extremely time-consuming for teachers. The time 
involved varied from 10 minutes per student at grade 1 to 25-30 minutes at grades 2-3. While 
some classes only required two days of assessment time, other teachers reported taking the entire 
first week to complete the assessments. During a school year, the test publisher suggests that 
teachers complete two assessments per day with the entire process taking three-four weeks. 

Because it was assumed that students who enrolled in the program were below grade level, 
some teachers reported starting too low on the assessment for some of the students. One teacher 
said that the assessment helped her to determine the reading level of her third-grade students who 
ranged from level 6 to level 44 (the highest). Some students who were assessed did not require 
remediation. One teacher stated that the program’s organizers should stress eligibility criteria if 
the program is used next summer. 

Some concern was expressed about the ability to show growth at the lower levels using this 
assessment tool. A teacher stated that the DRA “was a good measurement tool for those students 
who were ready to start learning reading strategies. ” However, she said, those students who 
were very low and did not know letters or sounds and who had not been exposed to this before did 
not show advancement on the DRA tool. “They did make advances and there was no assessment 
tool to show that growth. ” Growth for the non-readers might be expressed in an anecdotal note, 
suggested another teacher. One teacher said that “summer school is too short for the DRA to be 
an effective tool , but it could be in a regular school year. “ 

Many of the teachers suggested assessing students with the DRA prior to the beginning of 
summer school to “maximize instruction time. ” One teacher noted the merit of one assessment 
instrument for the district “so that moves from one school to another in the district would be 
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smooth for the students as well as immediate, accurate placement in reading by the receiving 
teacher. ” According to a third-grade teacher, “ The DRA is not easy, but it is a good tool. ” 

Attendance 

Teachers reported that attendance varied each day. Some parents had childcare problems 
that kept students at home. Other difficulties for parents involved catching the bus (for students 
who were not used to riding a bus), summer travel, lack of medication for students, and other 
activities planned for the summer. 

Several teachers suggested that a note might be given to parents at registration stressing 
attendance as very important to the effectiveness of the program. One teacher said, “It would help 
to make attendance a requirement for participation. ” 

Challenges Faced 

S.O.A.R. teachers were asked about the challenges they faced during the summer program. 
The following is a summary of the challenges they cited: 

• attendance and “wiggly” students; 

• correlating the DRA books to the Wright Group books; 

• an ESL student who could only read and write Spanish; 

• skill levels that were lower than materials purchased and the training provided; and 

• behavior management. 

Suggestions for Improving Summer Reading Program 

Teachers said that they would use the skills learned in the S.O.A.R. program when they 
return to their regular classroom. As one teacher stated, “The S.O.A.R. program has been an 
excellent program and very successful in my classroom. ” The following suggestions for future 
programs were made by teachers during the interviews: 

• include the Reading Recovery assessments for concepts of print and letter 
identification with the DRA for the first grade students; 

• debrief by grade level; 

• make the workshop specific to grade level; 

• build more planning time into the schedule; 

• provide more feedback about specific use of balanced literacy strategies; 

• find more ways to keep students on task; 

• recruit other volunteers for the classroom, especially those who are dedicated to 
coming several days each week; 

• inform UT professors that the program is going on so they can encourage students to 
participate; 

• let students and parents know what is expected in the way of attendance and behavior; 

• plan time for assessment prior to beginning of the summer program; 

• correlate books used (Rigby and Wright) to DRA and Reading Recovery levels; 

• increase the length of time for the staff development; 

• increase the program length to six weeks: 

• have PALM data available before summer school begins; 
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• allow a 10-15 minute break during the morning; and 

• use the Developmental Reading Assessment districtwide. 

Effectiueness of the Balanced Literacy Program 

All of the teachers interviewed believe that the balanced literacy program is an effective 
approach to teaching reading. One teacher said, “Both the teacher and the student know where 
the student is at all times. ” 

The S.O.A.R. teachers indicated that student progress was evident in this summer 
program, although many of the teachers believed that one or two more weeks of instruction would 
have a significant effect on accelerating literacy. The kindergarten students moving into first grade 
showed progress in recognizing letters, directionality of print, tracking letters and sounds, 
recognition of words, and fluency in putting word strings together, according to first-grade 
teachers. At grades 2-3, teachers reported gains in reading achievement and self-esteem for most 
students. 

Positive aspects of the S.O.A.R. program reported by teachers included the wide array of 
excellent materials, the dedicated staff, the low pupil-teacher ratio, improved student confidence in 
reading, and the overall learning experience that was provided. One teacher wrote, “The program 
was extremely beneficial - especially for those students who need small group/one-on-one 
instruction. The students seemed to feel confident and safe which enhanced their learning 
styles. ” Another teacher added that “this would be a good program for all classrooms. ” 

Principal Interviews 

The Title I evaluation staff interviewed the principals during the site visits that were 
conducted at each of the three S.O.A.R. campuses. A summary of the results of the principal 
interviews follows. Also included in this section are comments by the S.O.A.R. project director 
and curriculum specialist concerning suggestions for future summer programs. 

Training and Preparation 

In general, the site principals felt that adequate planning and preparation were provided for 
the teachers and themselves. However, teachers at one site were not able to get into their 
classrooms to set them up on the day set aside for these activities because the school was being 
painted. This resulted in teachers having to work over the weekend to organize their classrooms. 
Principals felt that at least an additional half day of paid preparation time for themselves and the 
teachers would have been useful. 

When asked about the balanced literacy training that was provided to teachers, the 
principals agreed that the sessions were very valuable. Several principals commented that more 
modeling of classroom behavior would have been appreciated, especially if the participants could 
have experienced an actual school day in “real time.” All agreed that the afternoon debriefing 
sessions were helpful for the teachers to work through problems encountered during the summer 
program. 

Preregistration did not provide principals with a good estimate of the number of students 
they would be serving at their campuses. Part of the problem, according to the principals, was that 
information was not provided to parents in a timely manner, resulting in some parents making other 
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plans for their children. However, given the timeframe in which the summer program was 
organized for this pilot year, the principals understood the problems with notification. One 
principal noted that parents might have been more likely to send their children to S.O.A.R. if an 
Extend-a-Care program had been available in the afternoon. 

Dai/y Operations 

When asked about the logistics of running the summer program, the principals agreed that 
the first several days were “busy. ” There were some problems with the bus routes initially, 
primarily because the drivers were unfamiliar with the routes. Also, students who do not normally 
take buses attended S.O.A.R., so it took them a few days to adjust to the new experience. Two of 
the summer sites had additional programs operating simultaneously, and there were some mix-ups 
with students not knowing which program they should be attending. Again, problems were 
generally resolved within a few days. 

According to the principals, teachers and students adapted well to the S.O.A.R. daily 
schedule. The program participants also like the half-day structure. However, one of the 
principals felt that the students, especially the youngest ones, would have benefited from “a few 
minutes outside. ” 

In terms of daily attendance, all of the principals reported that this presented a challenge. 
The parent training specialist at each campus worked daily to follow up on student absences with a 
telephone call to the child’s home. The principals indicated that they stressed the importance of 
regular attendance to the parents whenever possible through the phone calls, newsletters, and 
parent meetings. One of the principals noted that it would have been helpful “to have attendance 
tied to the program. ” 

When asked about the challenges of using an unfamiliar facility, the principals reported 
dissimilar experiences. One reported that there were no problems and that the host principal had 
helped to ensure that the experience would be a smooth one. Another reported that there were 
some problems initially sharing space with staff from other programs operating at the facility. The 
third principal reported that there were many challenges associated with the facility, primarily 
because of renovations that were ongoing for the duration of the S.O.A.R. program Better 
coordination between the host principal, the custodial staff, and the S.O.A. R. staff would have 
helped alleviate many inconveniences. 

In terms of necessary accommodations for bilingual, special education, or high-achieving 
students, the principals reported that the teachers encountered few problems. For the most part, 
special education students were indistinguishable from other S.O.A.R. students, so few 
accommodations were necessary. One campus had several monolingual Spanish-speaking students 
enroll and the Spanish S.O.A.R. materials were too advanced for their skills, so other materials had 
to be used. One of the sites transferred several of their bilingual students to a bilingual summer 
school program that was operating at the facility to better meet the students’ needs. All of the 
principals reported that higher-achieving students participated in literacy activities that better 
addressed their skill levels. In one case, more advanced students worked with the S.O.A.R. 
librarian, using materials tailored to their needs. 
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Materials 

With a few exceptions (e.g., tape dispensers, tape recorders), the office supplies provided 
for the S.O.A.R. program were reported to be adequate at all sites. The principals all made 
favorable comments about the balanced literacy instructional materials purchased for the summer 
program, and one noted that it would be helpful to be able to use the materials during the regular 
school year. Two of the principals commented that there were not enough instructional materials 
for the lowest literacy levels, however. 

When asked about the effectiveness of the DRA as an assessment tool, the principals 
generally agreed that it was a good choice. All felt that the assessment process improved as 
teachers became more familiar with the materials. One principal suggested that it would be 
helpful, if the materials are used again, “to try to find a way for the assessment tools used in 
Rigby , Wright Group , Celebration Press , Reading Recovery , and Balanced Literacy to go on the 
same scale. If the district adopts this program they should be aware of the difference between 
the scales for Reading Recovery and DRA. ” 

Support Staff I Activities 

Each S.O.A.R. site was assigned a parent training specialist, a nurse, a librarian, one 
secretary, and classroom monitors/teacher aides to assist with the summer program. One principal 
reported that the primary responsibility of the librarian was “to place an appropriate level book 
into each child's hand. ” In all cases, the librarians worked with the classroom teachers to enhance 
the students’ learning experiences. The librarians frequently came into the classrooms and read 
aloud to students. In other cases, they worked with the higher-achieving students in literacy 
activities appropriate to the students’ skill levels. The librarian at one site served as a resource for 
parents, helping them to select appropriate materials to check out of the school’s library for use at 
home. 

When asked specifically about parental involvement with S.O.A.R, all of the principals 
reported that the parent training specialist was a valuable asset to the program. Student attendance 
was monitored with the assistance of the parent training specialist at each site. One of the 
principals reported that a weekly newsletter was sent to parents to keep them informed about the 
progress of the program. All of the principals indicated that successful parent meetings had been 
conducted at each site. At one location, an introductory meeting was held to get to know the 
parents and to introduce word building to them. At another campus, an evening parent meeting 
was attended by over 60 people. This meeting was arranged like an open house, with student 
presentations and visits to the classrooms where teachers gave parents an overview of balanced 
literacy. At the third site, both morning and evening meetings were scheduled for the parents. 
According to the site principal, “during the meetings we reinforce the need for children to read 
between the end of the S.O.A.R. program and the beginning of school. We also stress the ease , 
enjoyment and benefits of reading to your child. ” 

Program Feedback 

The principals reported that they had received very positive feedback about the S.O.A.R. 
program from their teachers. According to one principal, the teachers “feel the environment is 
sane , the materials are ample and the class size is just right. 39 However, it was noted that the 
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teachers expressed some concern about administering the DRA while students were still becoming 
familiar with the concept of working in centers. 

When asked about implementing S.O.A.R. strategies during the regular school year, the 
principals expressed some concerns about how well this will work. The materials are expensive, 
according to one principal, so they would need to be purchased over time. Consequently, 
something like the S.O.A.R. program would need to be integrated slowly into the regular program. 
Another principal commented that it would be beneficial to train all teachers in the district in 
balanced literacy strategies. One of the site principals noted that “after we go back to the world of 
22 students in a class, we can talk about balanced literacy and centers in order to preserve these 
strategies, but 1 don ’t know how well it will work. ” 

Suggestions for Improving Summer Reading Program 

The principals were asked if they had any suggestions for improving the summer school if 
the district used this plan next year. In comments similar to ones made by teachers, the principals 
said they would like to see the parents notified earlier about their child’s involvement in the 
program. Two of the principals suggested that there should be time set aside at the beginning of 
the session for DRA assessments only. If necessary, “add a week, or maybe just three days, to 
the schedule so that testing could be done and logistics worked out and still have 19 days to 
teach. ” Another principal suggested that it would be helpful to provide a lead Balanced Literacy 
teacher to work with classroom teachers instead of the principal. “Lead teachers can model 
techniques much better than principals. Principals are usually tied up in their specific areas, 
and sometimes cannot get back into the classroom mode. ” 

Two of the principals suggested that it would be useful to coordinate with other programs, 
such as Reading Is FUNdamental, to get books students can take home. One principal would like 
to arrange a field trip to a local bookstore at the end of the program so students could purchase a 
book of their own with funds donated by an adopter. 

Several of the principals’ suggestions were echoed by the S.O.A.R. project director and the 
curriculum specialist. They agreed that the program’s tuneline was “ rushed ” this year, and they 
would hope to “communicate in a more timely manner to schools, principals, teachers, and 
parents” if the summer program is organized for next year. According to Terry Ross, the project 
director, “the timeline for S.O.A.R. was incredibly short, and we can certainly do a better job on 
that” if the program is offered in the future. Ms. Ross and Kathryn McKenzie, the curriculum 
specialist, also recognized the difficulty of completing DRA assessments while the students are 
becoming familiar with classroom procedures, and suggested that either the assessments be 
completed prior to the start of the program or have summer school teachers work in teams until all 
of their students are assessed. 

Regarding the recruitment of site principals for S.O.A.R., both the project director and the 
curriculum specialist would like to be more involved in the hiring process in the future. They 
indicated that the site principal positions should be posted and the applicants should be screened 
carefully for appropriate qualifications. The principal is a critical element in the success of any 
school program, and it is essential that the persons selected to head the S.O.A.R. program at each 
site should be knowledgeable in balanced literacy and the mechanics of running a school. 
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When asked about the hiring and training of teachers for S.O.A.R., Ms. Ross and Ms. 
McKenzie indicated that they would like to be more involved here as well. They would like to start 
with a cadre of trained teachers to team with new teachers if the program is offered next summer. 
Also, the training sessions should be longer to provide more in-depth coverage of critical balanced 
literacy concepts. 

Finally, all of the individuals who were interviewed for this evaluation commented that the 
summer program should be extended in the future. Most of the people involved in S.O.A.R. felt 
that six weeks would be a more reasonable length for the program, and that 19 days was not 
adequate in many cases to cover all of the material and to make a significant difference in the 
reading skills of the participants. 
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Summary 

This evaluation of S.O.A.R. has provided information on the participants in the summer 
reading program, A review of the literature related to reading instruction was presented, along 
with an overview of the evaluation methodology. This was followed by a discussion of the 
qualitative and quantitative data analyses that were undertaken for the evaluation. The information 
gained from interviews with principals and teachers was summarized, along with the results from 
teacher surveys. Student and teacher demographics were presented. Finally, achievement data 
from the Developmental Reading Assessment were analyzed. 

Demographic and Achievement Data 

Of the 388 students who attended at least five days of the S.O.A.R. program, 55% were 
male. Forty-seven percent of these students were Hispanic, 37% African American, and 16% were 
Anglo/Other. Students from 37 AISD schools and four private schools participated in the 
program. Twenty-five percent of the students will enter grade 1 in 1998-99, 37% grade 2, and 
38% grade 3. 

Forty-five teachers taught reading in the S.O.A.R. classrooms across the three campuses. 
Of these teachers, 57% were Anglo/Other, 23% were African American, and 20% were Hispanic. 
Only two of the 45 teachers were male. The majority of the teachers taught core academic subjects 
at the elementary level during the 1997-98 school year. As a group, the participating teachers had 
an average of 8.7 years of teaching experience with AISD. 

During the four-week S.O.A.R. program, 85% of the participating students showed 
reading improvement by advancing one or more levels from pretest to posttest on the DRA, the 
assessment instrument used in the program. The average number of levels gained by students, 
regardless of the number of days of attendance, was 1.9, with a range from zero to 10 levels. 
Fifty-two students (13%) advanced from the lowest level on the pretest to a higher level on the 
posttest. In terms of advancement between stages on the DRA, 111 students moved one of more 
stages during the course of the summer program. 

Attending at least 17 days of the S.O.A.R. program had a positive effect on student gains. 
When looking at all students, regardless of the number of days of attendance, the average gain in 
level was 1.9, compared with an average gain of 2.03 levels for students who attended at least 17 
days. A similar pattern was seen in terms of the average stage gains. However, because the 
average attendance rate overall was 16.3 days, the achievement differences between all students 
and those with at least 17 days of attendance would be expected to be similar. 

When scores were examined by grade, students at Grade 2 showed the greatest mean gain 
in both level and stage. The mean gain in levels was 2.4 and the mean gain in stages was 0.5 for 
grade 2. 

Surveys and Interviews 

As a result of the evaluation efforts, it was determined that the majority of the staff 
involved with S.O.A.R. was highly in favor of the program. Teachers expressed strong support for 
the balanced literacy approach to reading and indicated their willingness to use the strategies 
learned from the program when they return to their regular campuses. However, they did express 
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concern that suitable instructional materials were not available for students at the lowest literacy 
levels. 

Principals indicated that there were some problems encountered with implementing the 
program initially, but these were generally resolved within a few days. In terms of the training 
provided to teachers, the principals agreed that the sessions were valuable but that more modeling 
of classroom behavior would have been helpful. The principals also indicated that they had 
received very positive feedback about the S.O. A.R. program from their teachers. 

When asked about improving the summer reading program if it is offered again in AISD, 
teachers and principals made the following suggestions: 

• Notify parents earlier about their child’s involvement in the program. 

• Extend the program to six weeks in length. 

• Carefully monitor the enrollment process, stressing eligibility for the program and the 
importance of regular attendance. 

• Organize the pretesting schedule so that it does not interrupt instructional time. 

• Correlate the books used for the DR A with Reading Recovery levels. 

• Start with a cadre of trained teachers to team with new teachers. 

Recommendations 

As a result of reviewing the evaluation findings, the following recommendations 
concerning the S.O. A.R. program are offered for consideration: 

• Continue S.O. A.R. next summer, and incorporate elements of the balanced literacy 
approach into regular classroom teaching throughout the year. 

• Lengthen the program to six weeks to maximize the benefits of instruction during the 
summer break. 

• Maintain a low teacher-pupil ratio for the summer classes. 

• Enforce a stricter enrollment policy for students, one that stresses the eligibility 
requirements for the program and the importance of daily attendance. 

• Purchase additional materials that better meet the needs of pre-reading students. 

• Increase the emphasis districtwide on balanced literacy as an approach to teaching 
reading by offering professional development for teachers. 

• Recruit teachers for the summer program who are likely to use the balanced literacy 
training in their classrooms during the school year. 

• Recruit site principals for the program who are knowledgeable in balanced literacy and 
the mechanics of running a school. 

• Use S.O. A.R. teachers to train other teachers in balanced reading so they can use the 
strategies in their classrooms. 

• Increase the length of the training required of teachers before the summer program. 

• Assess reading levels of program participants with the DRA before the summer 
program begins. 

• Continue to monitor achievement in future summer programs. 
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Appendix A: Survey and Interview Instruments 

S.O.A.R. Teacher Interview 

1. Did the schedule work for you or did you have to make adjustments? 

2 . Was the Developmental Reading Assessment a useful tool? 

3. How long did the assessment process take? 

4. Did you have enough materials for an effective program? 

5. Have you previously used Reading Recovery strategies? 

6. Did the training prepare you for the balanced literacy approach? 

7 . Was attendance a problem for any of your students? 

8. What challenges have you encountered during this summer school program? Was 
debriefing helpful in addressing problems? 

9. Do you feel that students are making progress during this 4-week period? 

10. Do you think that this is an effective method for teaching literacy? 

11. What skills and/or knowledge will you take back to your classroom from this program? 

12. Do you have any suggestions for improving the summer school if the district uses this 
plan next year? 
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S.OAR. Teacher Survey 

1 . The staff development I received adequately prepared me to teach in the S.O.A.R. program. 

a) Strongly Agree d) Disagree 

b) Agree e) Strongly Disagree 

c) Unsure 

2 . I received enough information during training to feel confident that I administered the 
Developmental Assessment System correctly in my classroom. 

a) Strongly Agree d) Disagree 

b) Agree e) Strongly Disagree 

c) Unsure 



3. I feel that the Developmental Assessment System was a good instrument to measure student 
growth during the S.O.A.R. program. 

a) Strongly Agree d) Disagree 

b) Agree e) Strongly Disagree 

c) Unsure 



4. The daily schedule contained an adequate mix of activities to keep all students engaged 
academically throughout the day. 

a) Strongly Agree d) Disagree 

b) Agree e) Strongly Disagree 

c) Unsure 

5. The daily schedule and the curriculum worked together to help ensure that student 
progress was achieved in the S.O.A.R. program. 

a) Strongly Agree d) Disagree 

b) Agree e) Strongly Disagree 

c) Unsure 



6. I feel that a balanced reading program provides useful instructional strategies to accelerate the 
literacy learning of students. 

a) Strongly Agree d) Disagree 

b) Agree e) Strongly Disagree 

c) Unsure 



7. I will use strategies and/or materials from the S.O.A.R. program in my regular classroom next 
year. 

a) Strongly Agree d) Disagree 

b) Agree e) Strongly Disagree 

c) Unsure 
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S.OAR. Principal Interview 

1. Do you feel that there was adequate planning and preparation time for yourself and your 
teachers before summer school began? 

2. Did the logistics of breakfast, buses, and lunch work well at your school? 

3. Did teachers and students adapt well to the schedule? 

4. Was pre-registration a good indicator of the number of students participating in the 
program? 

5. Were there problems with attendance? If so, how did you address the problems? 

6. What were the challenges associated with using an unfamiliar facility? 

7. Were there enough materials for all the classrooms? Did teachers use the materials 
effectively? 

8. Do you think that the DRA was an effective assessment tool for this program? 

9. Do you feel that teachers were adequately trained for the balanced literacy approach? 

10. How did your campus use the librarian’s services? 

1 1 . How did you involve parents with the summer program? 

12. Did your teachers have to make accommodations for bilingual, special education , or high- 
achieving students? If so, what accommodations were made? 

13. What kind of feedback have you received from teachers about the summer school 
program? 

14. Could this program be successfully implemented during the regular school year? Will you 
encourage your staff to use any of these strategies? 

15. Do you have any suggestions for improving the summer school if the district uses this plan 
next year? 
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Appendix B: Text Book Leveling Correlations 
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7 : 45 - 8:15 

8:15-9:00 



9:00-10:30 



10 : 30 - 11:15 
11:15 -11:30 

11 : 30 - 12:00 

12:00 

12 : 15 - 1:15 



Appendix C: S.O.A. R. Daily Schedule 

Breakfast 
Opening Routines 

Calendar 

Shared Reading 

Shared Writing/Message of the dag 
Guided Reading/Centers 

Rotate among centers 

Each group lasts approximately 20-25 minutes 
Writers Workshop 
Read Aloud 
Lunch 

Dismissal 

Debrief/Planning 
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Appendix D: Summer School Materials List (per teacher) 

GRADE I 

1 Letter Storage Box 
1 Magnetic Marker Board 
1 Set Magnetic Letters (uppercase) 

1 Set Magnetic Letters (lowercase) 

1 Kinderrimes Add-To-Package (Rigby) 

1 Kindergarten Add-To-Pack (Rigby) - used for centers - reading/writing 
1 Decodable Stories - Set A (Rigby) - to be shared by first grade team 
1 Daily News Pocket Chart 

1 Tape Recorder 

2 Foundations Level 1, Sets A-D (Wright Group) - set of 64 books to be divided among first grade 

teachers - for independent reading 

1 Shared Reading Value Set, Stage 1 (Rigby) - set includes 15 big books, 80 small books and 15 
audiocassettes 

GRADE 2 

1 Letter Storage Box 
1 Magnetic Marker Board 
1 Set Magnetic Letters (uppercase) 

1 Set Magnetic Letters (lowercase) 

1 1 st Grade Add-To-Pack (Rigby) used for centers - reading/ writing 
1 Daily News Pocket Chart 
1 Tape Recorder 

1 Decodable Stories - Set B/Set C (Rigby) - to be shared by second grade team - take home books 

2 Foundations Level 1, Sets E-J (Wright Group) - six each of 104 pupil books to be shared by second 

grade teachers - for independent reading 

1 Shared Reading Value Set - Stage 2 (Rigby) - set includes 15 big books, 80 small books and 15 
audiocassettes 

1 Shared Reading Value Set - Stage 3 (Rigby) - set includes 15 big books, 80 small books and 15 
audiocassettes 

GRADE 3 

1 Letter Storage Box 
1 Magnetic Marker Board 
1 Set Magnetic Letters (uppercase) 

1 Set Magnetic Letters (lowercase) 

1 Daily News Pocket Chart 
1 Tape Recorder 

1 Worlds of Poetry Library (Sadlier-Oxford) 

1 Second Grade Add-To-Pack (Rigby) - used for centers - reading/writing 

1 Decodable Stories - Set C/D (Rigby) - to be shared by third grade team - take home books - decodable 

blackline masters 

2 Foundations Levels 2-5 (Wright Group) - six each of 69 pupil books to be shared by third grade 

teachers — for independent Reading 

1 Shared Reading Value Set - Stage 4 (Rigby) - set includes 15 big books, 80 small books and 15 
audiocassettes 

1 Shared Reading Value Set - Stage 5 (Rigby) - set includes 15 big books, 80 small books and 15 
audiocassettes 

* 10 sets of Developmental Reading Assessment Packages were ordered for each campus - to be divided 
between grade levels 
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